The invention relates generally to methods and apparatus for interpolating image information to, for example, reconstruct missing pixel information and/or missing lines in a video field or other set of pixels using existing pixel information, and more particularly to methods and apparatus for interpolating image information associated with diagonal angles or other specific geometric feature in an image.
Interpolating image information to change existing pixels or generating missing pixels, for example missing lines or individual pixels using interpolation techniques are known in the printing field and screen display field. Interpolation can be used for many purposes including, but not limited to, scaling an image or portion of an image in size up or down in any suitable direction, speeding up the processing by utilizing fields of video to produce frames from information from a field or for any other suitable purpose. Deinterlacing of even and odd fields of interlaced video fields to produce frames of video is also known. Known solutions include, for example, spatial domain deinterlacing techniques such as bob techniques which interpolate missing lines or pixel information from existing pixels that are present, for example, in a video field using averaging techniques that may include the intensity information of one or more surrounding existing pixels in existing lines above and below a particular position.
Weaving and other temporal domain deinterlacing methods are also known which employ, for example, interpolating missing pixels by averaging or otherwise combining pixel information over a number of sequential and non-sequential fields or frames. Also, different blocks of pixels in a frame may be interpolated using bob or weave techniques or other techniques.
A bobbing method may average two closest pixels to reconstruct missing lines but can create jagged looking diagonal lines. Other spatial domain methods may calculate orientation for example, by finding correlation between diagonal pixels and interpolate between the most correlated pixels. However, such methods may have a high diagonal orientation selection error unless sophisticated decision making techniques are employed. In addition, a wide kernel of pixels (a kernel may be for example, a block of nine pixels in a 3×3 array or any other suitable configuration) may be needed to determine shallow angles but the orientation selection can be more prone to error. When a wrong orientation angle is selected, it may generate undesirable artifacts in the output image, whether output on paper or display screen.
Other types of interpolation such as motion adaptive deinterlacing and motion compensated deinterlacing are also known. Motion adaptive (MA) deinterlacing detects motion between consecutive video fields and takes the motion into account when interpolating information. Motion detection, in its simplest form, is accomplished using inter-field differences (either differences between fields of same polarity, consecutive fields (of opposite polarity) or a combination of both). In areas of detected motion, typically motion adaptive deinterlacer falls back to spatial deinterlacer, which is an interpolator utilizing only the pixels in the spatial vicinity of the pixel to be interpolated. Motion compensated (MC) deinterlacers use extracted motion vectors of a field to interpolate along motion trajectories. In both cases, spatial deinterlacing is typically used as a fall-back mechanism. Without falling back to spatial deinterlacing, MA and MC would suffer weaving artifact, which is very objectionable to the observer. Spatial-only deinterlacer itself would produce viewable result, though might be perceived as lack of detail and/or flickering in still area of the video.
Gradient square tensor operations are also known and are typically used to perform edge detections to detect, for example, line edges or edges of other portions of images in a frame, for example and may be used for pattern recognition purposes. However, such edge detection techniques are not known to be used in connection with, among other things, pixel interpolation.
Pixel interpolation methods may be employed in any suitable application and may be employed, for example, by digital televisions, cell phones, cameras, camcorders, printers or any other suitable device and may be implemented using one or more processors such as CPUs, video graphics coprocessors, discrete logic, or any other suitable structure.
Where the device requires battery power, such as handheld devices or other portable devices, it may be desirable to provide a relatively accurate and uncomplex mechanism to save processing resources and reduce battery power consumption. In addition, it may be desirable to provide a low complexity and relatively accurate interpolation technique that may be suitably applicable to diagonal angles or other geometric features that are detected in an image such as a frame, field, or other suitable collection of pixel information.
For purposes of background information, FIG. 1 illustrates a portion of an image 10 wherein the pixels marked “X” are missing pixels and are reconstructed from original pixels which are marked as “O”. As such, the diagram illustrates a portion of an image that contains missing pixels prior to interpolation and final image generation. If the missing pixels are reconstructed, the image resolution is doubled both horizontally and vertically. Conventional methods for interpolating the missing pixels include, for example, bilinear interpolation, polyphase filters, spline-interpolation such as bicubic interpolation, and other suitable methods. However, such methods typically do not consider local texture orientation and typically do not reconstruct diagonal lines properly.
FIGS. 2-4 illustrate three types of missing pixels or pixels to be interpolated. FIG. 2 shows the pixel to be interpolated to be in the middle of two horizontal original pixels, FIG. 3 shows the pixel to be interpolated to be in between two vertical original pixels and FIG. 4 illustrates a pixel to be interpolated that is in the center of a square formed by four adjacent original pixels.
For a pixel to be interpolated, the local area surrounding the pixel of interest may exhibit a certain specific geometric feature such as a line, an edge, or a slope orientation being diagonal, so that the pixel to be interpolated may be suitably reconstructed by averaging pixels over the pixels along a line that closely follows the orientation. FIG. 5 illustrates a number of possible orientations for the missing pixel of a group of a pixel shown, for example, in FIG. 4. Once an orientation is determined, a missing pixel value can be calculated, for example, as the average of two original pixels that tap the ends of the line. For example, if the pixels 12 and 14 are determined to be in a geometric feature such as in a diagonal line, the pixels 12 and 14 can be averaged to determine the missing pixel X shown as missing pixel 16. However, if the local area does not exhibit any type of specific geometric feature, the missing pixel value may be an average of the nearest four pixels in this example. However, it would be desirable to have a circuit, apparatus and methods to provide improved image interpolation.